Decentralized Estimation using distortion sensitive learning vector quantization

A typical approach in supervised learning when data comes from multiple sources is to send original data from all sources to a central location and train a predictor that estimates a certain target quantity. This can be inefficient and costly in applications with constrained communication channels, due to limited power and/or bitlength constraints. Under such constraints, one potential solution is to send encoded data from sources and use a decoder at the central location. Data at each source is summarized into a single codeword and sent to a central location, where a target quantity is estimated using received codewords. This problem is known as Decentralized Estimation. In this paper we propose a variant of the Learning Vector Quantization (LVQ) classification algorithm, the Distortion Sensitive LVQ (DSLVQ), to be used for encoder design in decentralized estimation. Unlike most related research that assumes known distributions of source observations, we assume ∗Corresponding author, tel: (215) 204 7230, fax: (215) 204-5082 Email address: [email protected], [email protected] (Mihajlo Grbovic and Slobodan Vucetic) Preprint submitted to Pattern Recognition Letters August 16, 2014 that only a set of empirical samples is available. DSLVQ approach is compared to previously proposed Regression Tree and Deterministic Annealing (DA) approaches for encoder design in the same setting. While Regression Tree is very fast to train, it is limited to encoder regions with axis-parallel splits. On the other hand, DA is known to provide state-of-the-art performance. However, its training complexity grows with the number of sources that have different data distributions, due to over-parametrization. Our experiments on several synthetic and one real-world remote sensing problem show that DA has limited application potential as it is highly impractical to train even in a four-source setting, while DSLVQ is as simple and fast to train as the Regression Tree. In addition, DSLVQ shows similar performance to DA in experiments with small number of sources and outperforms DA in experiments with large number of sources, while consistently outperforming the Regression Tree algorithm.